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DETAILED ACTION 

EXAMINER'S AMENDMENT 

1 . An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 
1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 
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1, (Currently amended) A display system for at device, computing: 

cornp utei" e*ec*.jtshie \r I u ! r-; mMe to; 

receive images from a first sensor and a second sensor representing 

scenery outside the device; 
defect moving obpols in the Images- 
fuss the images to a single viewpoint: 

transform the fused image to a first viewpoint image from a fmt 

operator station in the device and a second viewpoint image 
from a second operator station in the device, wherein the first 
and second viewpoint images conform to the scenery outside tile 
device from each operator station; and 

generate a third display area associated with at least two mutually 

exclusive windows of information on a display device for the first 
operator station, the display device is mounted in an instrument 
panel of the device and the third display area and the two 
mutually exclusive windows of information are presented on the 
display device; 

generate a third display area associated with at least two mutually 

exclusive windows of infomiation on another display device for 
the second operator station, the other display device is mounted 
in the instrument panel of the device and the third display area 
and the two mutually exclusive windows of information are 
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2, (Original) The display system of Claim 1 t further comprising; 

computer executable Instructions operate to: 

combine the firs! and second viewpoint images with symbols, wherein 
the symbols represent information regarding the operational 
state of the device ami trio moving objects detected in the 
images. 



3, (Original) The display system of Claim 1 , wherein the instructions for 
detecting moving ejects in the first sensor image are configured to execute in a fret 
processor, and the instructions for detecting moving objects in the second sensor 
image are configured to execute m a second processor simultaneously wth the 
instructions in the first processor. 



4. (Ordinal) The display system of Cia*m 3, wherein the instructions for 
to the first viewpoint image are configured to exec 
for transforming the fused image to the 
to execute in tl 



5. (Original) The display system of Claim 2, wherein the symbols 
the moving objects in the vicinity of the 




7. (Original) The display system of Claim 2 S wherein the symbols 
zards in the vicinity of the device, 

8, {Original) The display system of Claim 2, wherein the computer executable 
instructions are farther operable to receive an enhanced image from: a third sensor 
configured to provide an image of Hie out-the-wlndow scenery in low-visibility 



Application/Control Number: 10/615,634 
Art Unit: 2628 



Page 5 



9. (Original) The display system of Claim 8, wherein the eompuier&s 
ciiom are further operable to fuse the single viewpoint Image with the 



10. (Original} The display system of Claim 9, wherein the computer 
fable instructions ere further operable to utilize data from at least one position 
* to determine- the location of She objects with respect to the device. 



i 1 > {Original) The display system of Claim 2, wherein the computer 
executable instructions are further operate to utilise data from off-board dati 



12, (Original) The 



13, (Original) The display system of Claim B, wherein the third sensor k 
RADAR sensor. 



14. (Original) The display system of Claim 8, wherein the third sensor Is a 



15. (Canceled) 

18. (Currently amended) A method for providing an out-the-window visual 
scene on a display device, comprising; 

receiving an i mage of a portion of the out-the-wir«iow vi sual scene from the 

viewpoint of a first type of sensor; 
receiving another image of a porta of the ouMtowtefaw visual scene from 

the viewpoint of another of the first type of sensor; 
fusing the Images from the f rat type of sensors into a combined image to 
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transforming the fused Image to a first operator viewpoint and to a second 
operator viewpoint; and 

outpytting the first operator viewpoint image to a first display device and the 
second operator viewpoint image to a second display device, wherein 
the display devices are positioned in an instrument panel to provide the 
portion of a desired ouMhe window visual scene in combination with a 
window thai provides another portion of the desired ouMfca-window 



visual scene, and the viewpoint images are aligned with and scaled to 
conform to the out~tf*e-window visual scen e: and 




17. (Original) The method of Claim 16, further comprising detecting objects in 
the first fused image from the first type of sensor. 

18. {Original} The method of Claim 17, further comprising combining the first 
fused image wrth symbols representing the objects, 

19. (Original) The method of Claim 18, further comprising transforming the 
first operator viewpoint image and the second operator viewpoint image to conform to 

20. (Original) The method of Claim 18, further comprising fusing the first 
fused ismage with an enhanced image of a portion of the ©ut-the-winoow scenery from 
at least one of the group of a RADAR sensor and a FUR sensor, to generate a 
second fused image. 
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21 . (Original) The method of Claim 18, further comprising fusing the second 
fused image with an enhanced image of a portion of the euMhe-windew scenery from 
at least one of the group of a RADAR sensor and a FUR: sensor, to generate a 
second fused image. 

22. (Original) The method of Claim 21 > former comprising: transforming the 
second fused image to the first operator viewpoint and to the second operator 
viewpoint 

23, (Original) The method of Claim 20 $ further comprising.; providing portions 
of the transformed image with data from a terrain map database, 

24, (Currently amended) A device, comprising: 
a display device; and 

a display processor operable to; 

receive a first sensor image representing a portion of scenery outside 
the device; 

transform the first sensor image to a viewpoint image from an operator 
station in the device, wherein the viewpoint image is sized and 
oriented to conform to the scenery outside the device from the 
operator station; and 

output the first operator viewpoint Image to the display device, wherein 
the display device is positioned in an instrument panel to provide 
the portion of a desired out-the window visual scene in 
combination with a window that provides another portion of the 
desired ouMhe-window visual scene, and the viewpoint image is 
aligned with and seated to conform to toe out-the-window visual 
scenei j 
wherein the display processor is further operable to generate a common 

#£.j?!eY,,afea^ 
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can be customized fcv t->- . \ *■ mot to diss t. 



25. (Original) The device of Claim 24, wherein the 
operable to combine the viewpoint image with symbols, wherein 

te of 



28. (Original} The device of Claim 24, wherein the display 
operable to detect moving objects in the first sens 



processor is further 
the symbols 



27. (Original) The device of Claim 24 s wherein the display processor is 
further operable to generate symbols representing moving objects in the sensor 
Image end the operational state of the device* 

28, (Original) The device of Claim 24* wherein the display processor is 
further operable to generate symbols representing weather hazards in the vising of 
the device. 




30, (Original) The device of Claim 29, wherein the display processor is further 
operable to fuse the viewpoint image with the enhanced image. 

31 , (Original} The device of Claim 26, wherein the display processor is further 

32, (Original) The device of Claim 26, wherein the display processor is further 
operable to utilize date from off-board data sources regarding the objects, 
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33, {Original) The device of Claim 24, wherein the sensor to a video earners 

$4, (Original) The device of Claim 2S. wherein the second sensor is a 
RADAR sensor. 

35, (Original) The device of Claim 29, wherein Hie second sensor to a FUR 

36. (Canceled) 



37, (Currently amended) An aircraft comprising: 

a crewstation with cockpit windows; 

a first display device for one crewntember; 

a second display device for another crewmernben and 

a display processor operable to 

receive an Image of an ouMhe-vwndow visual scene from the viewpoint 
of a first type of sensor: 

receive another image of a portion of the out-the- window visual scans 



from the viewpoint of another of the first type of sensor; 
fuse the images from the first type of sensors into a combined image to 




output the first operator viewpoint image to the first display device and the 

second operator viewpoint image to the second display device, wherein 
the display devices are positioned in an instrument panel to provide the 
portion of a desired out»the window visual scene in combination with a 
cockpit window that provides another portion of the desired ouMha- 
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window visual scene, and the viewpoint Images are aligned with and ! 

scaled to conform to the ouMbe-wjndow visual scene; j 

wNsrern - the^jspliy..5j^mser.fe further ocer^ j 

&8£$m.mm associate d with at tea; t * ytu . * > winefcyws of I 
jnfonnitiofi: on eaAajspjay devi ce, .Mserein th e .common display, .area 

cmMmiomized by the operator tojQs&ia y , delated informa l | 
MiMJa.lb^ information displayed In the a ssociated w indows. 

38. (Original) The aircraft of Claim 37, wherein the display processor is 
further operable to detect objects in the first fused Image from the first type of sensor, 

30. {Original} The aircraft of Claim 3B« wherein the display processor Is 
further operable to combine the first fused Image with symbols representing the 
objects and primary flight information for the aircraft. 

40, (Canceled.) 

41 , (Original) The aircraft of Claim 37, wherein the display processor is 
further operable to fuse the first fused image with an enhanced image of a portion of 
the ouWhe-window scenery from at least one of the group of a RADAR sensor and a 
FUR sensor, to generate a second fused image. 

42. (Original) The aircraft of Claim 41 , wherein the display processor is 
further operable to fuse the second fused image with an enhanced image of a portion 
of the ouMhe-window scenery from at least one of the group of a RADAR sensor and 
a FUR sensor, to generate a second fused Image. 

43. (Original) The aircraft of Claim 4% wherein the display processor Is 
further operable fo transform the second fused image to the First operator viewpoint 
and to the second operator viewpoint 



Application/Control Number: 10/615,634 
Art Unit: 2628 



Page 1 



1 



44, (Original) The aircraft of Claim 43 f wherein the display processor is 
further operable to provide portions of the transformed Images wm$ data from a 
terrain map database. 

45. (Original) The aircraft according to Claim 3? further comprising: 

& terrain database coupled to provide the display processor with at bast a portion of 
the required ouMha-winrfow field of view on the display device, 

46- (Original) The aircraft of Claim 37, wherein the display processor ts 
further operable to display one of the operator viewpoint displays to the operator 
acting as pilot in command of the aircraft during a predefined aircraft operational 
state, and to allow the pilot in command to choose an option on the display device to 
view detailed information about the aircraft and aircraft subsystems during other 
aircraft operational states. 



47, (Canceled) 
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Authorization for this examiner's amendment was given in a telephone interview with 
Mary Jo Bertani on 8/8/2007. 

Allowable Subject Matter 

Claims 1-14, 16-35, 37-39, and 41-46 are allowed. 

The following is an examiner's statement of reasons for allowance: the cited prior art does not 
disclose or render obvious the combination of elements recited in the claims as whole. 
Specifically, the cited prior art fails to disclose or render obvious the following limitations: 

As per independent claim 1, the claimed: 

"wherein the first and second viewpoints images conform to the scenery outside the device", 
"generate a third display area associated with at least two mutually exclusive windows of 
information on a display device", "wherein the third display areas can be customized 
independently", and "first and second viewpoint images are displayed on the respective display 
devices". 

As per independent claims 16, 24, and 37, these claims are also allowed for the same reasons as 
independent claim 1 . 
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Any comments considered necessary by applicant must be submitted no later than the payment 
of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. 
Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 

Related Prior Art 

US Patent 6,809,407 (Kulas): teaches of an out-of-the-window display that conforms with the 
outside scenery in figure 3 on display screen 102. However, the claims for this application are 
novel over Kulas, because the invention, as claimed, displays the viewpoint images conforming 
with outside scenery on the display in conjunction with the claimed: "third display area 
associated with at least two mutually exclusive windows of information on a display device" 
wherein the third display areas can be customized independently. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Daniel F. Hajnik whose telephone number is (571) 272-7642. 
The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Ulka Chauhan/ 

Supervisory Patent Examiner, Art Unit 
2628 

/Daniel Hajnik/ 
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